Apparatus using one optical sensor to recover audio information from analog and digital soundtrack carried on motion picture film

ABSTRACT

An apparatus recovers audio information from analog and digital motion picture film soundtracks using a single optical sensor that senses across the widths of both soundtracks. Symbols representing audio information in the digital soundtrack are encoded in two dimensions and are recovered by oversampling the symbols in two dimensions. The two-dimensional encoding of digital information can be used with a variety of recording media; however, in one embodiment, the symbols are carried on motion picture film between the sprocket holes.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation of U.S. patent application Ser. No. 07/937,887filed Sep. 30, 1992, now U.S. Pat. No. 5,544,140 resulting frominternational patent application No. PCT/US 92/00898, which is acontinuation-in-part of U.S. patent application Ser. No. 07/710,174filed Jun. 4, 1991, now abandoned, which is a continuation-in-part ofU.S. patent application Ser. No. 07/650,571 filed Feb. 4, 1991, nowabandoned.

TECHNICAL FIELD

The invention relates generally to a storage medium carrying symbolsrepresenting digital information, the recovery from a storage medium ofsymbols representing digital information and the determination of thedigital information represented by such symbols. More particularly, theinvention relates to storage media carrying the symbols in twodimensions and to the recovery of such symbols by oversampling in twodimensions. While the invention has many applications, the invention isdescribed in connection with preferred embodiments in which the symbolsare carried by and recovered from an optical-storage medium, namely,motion picture film, the symbols representing digital information intowhich motion picture soundtrack and related information is encoded.

BACKGROUND ART

The last two decades have seen an explosion of interest in motionpicture sound, in large part triggered by the ever-increasing quality ofhome high-fidelity systems. There is a continuing interest in new andimproved motion picture soundtrack release formats.

Most films today are released with either conventional monophonic("Academy") optical soundtracks or with stereo-variable-area (SVA)optical soundtracks with analog noise reduction, which are most widelyknown under the trademark "Dolby Stereo." "Dolby" and "Dolby Stereo" aretrademarks of Dolby Laboratories Licensing Corporation.

The Academy format, originated in the 1930's, suffers from an extremelypoor frequency response. Even though the soundtrack itself may extend tobeyond 8 kHz, other elements within the recording and playback chainassociated with the format restrict the bandwidth such that the audiencewill hear very little above 4 or 5 kHz. In addition, the Academy formathas relatively high distortion, a barely adequate signal-to-noise ratioand perhaps the greatest shortcoming of all: it is only monophonic(mono).

Dolby Stereo is possibly the first motion picture optical soundtrackformat which can truly be called high fidelity, and which is availableto all theaters. Even though magnetic soundtracks (on 35 mm and 70 mmfilm) as long ago as the 1950's had moderately acceptablespecifications, the high release print costs were in large partresponsible for very few audiences ever hearing the magnetic versions.Stereo optical prints, on the other hand, have no premium cost aboveconventional optical soundtracks, and this has resulted in wideavailability. A large majority of Dolby Stereo films are released"single inventory"--that is, a separate monophonic Academy print is notreleased because the producer considers the Dolby Stereo film to provideacceptable audible compatibility when played in Academy mono equippedtheaters.

The Dolby Stereo optical soundtrack format provides four channels ofinformation (left, center, right and surround) matrix encoded onto thetwo SVA optical film soundtracks. The original Dolby Stereo formatemploys Dolby A-type analog audio noise reduction. In the mid-1980'sDolby Laboratories introduced an improved analog audio processingsystem, Dolby SR, and that system has been applied to many Dolby Stereofilms. The use of Dolby SR results in a dramatic improvement in volumerange and frequency response. Even in a quiet and well-equipped theater,optical print noise is below the theater's ambient noise floor, whileundistorted sound peaks are amply loud for most startling specialeffects. Frequency response extends to 16 kHz.

In spite of these advances in analog soundtrack fidelity, filmsoundtracks have long been considered a candidate for digital coding.Digital audio encoding has become integrated into the mainstream of bothconsumer and professional use. The Compact Disc has earned wide consumeracceptance. As a result, a digital soundtrack would benefit from thepopular conception that digital sound is inherently better, anundeniable added attraction at the theater box office.

In addition, a digital soundtrack may provide increased resistance toaudible degradation of the soundtrack caused by the wear and tear ofcommercial exhibition, and can diminish the audible effects of projectorwow and flutter. Multiple channels could be supplied on an opticalsoundtrack for both 35 mm and 70 mm print formats. The soundtrack'sfrequency and dynamic range specifications could exceed even that ofcurrent Dolby Stereo formats.

The recent announcements of two digitally-encoded optical soundtrackformats for 35 mm and 70 mm film, respectively, have reaffirmed theinterest which exists in using digital soundtracks to improve motionpicture sound in the theater. See "Digital Optical Sound on 35 mmMotion-Picture Film" by Syd Wiles et al, SMPTE Journal, November 1990,pp. 899-908 and "The Advent of Cinema Digital Sound" by Clyde McKinney,The Film Journal, August 1990, pp. 22 & 43 (a 70 mm system).Unfortunately, both of these formats locate the digital soundtrackinformation in the area formerly occupied by the analog soundtracks,making the these new digital formats incompatible with existing analogfilm formats and existing analog projection equipment.

DISCLOSURE OF INVENTION

In accordance with the teachings of the present invention, a newapparatus and method are provided for recovering symbols representingdigital information carried by a storage medium. In a further aspect ofthe invention, a new apparatus and method are provided for determiningthe digital information represented by the recovered symbols. Accordingto yet a further aspect of the invention, a new configuration of storagemedium is provided for carrying symbols representing digitalinformation.

The symbols may be any differentiable symbols capable of representingdigital information and capable of being carried in two dimensions by astorage medium. In its broadest aspects, the invention contemplates theuse of any medium capable of carrying differentiable symbols encodedtwo-dimensionally on the medium.

Many prior art techniques for digital information storage and recoveryrely upon control or timing information carried by the storage mediumapart from the digital information, referred to herein as "flags." Flagsinclude information that may (1) establish the relationship between timeand distance across a storage medium, (2) identify segments of digitalinformation, (3) provide a structure in which digital information may bestored, (4) establish the size and/or storage density of discreteinformation-carrying areas, or (5) establish the bounds and/ororientation of information-carrying areas. Some examples of these flagsinclude so-called timing tracks, track indexes on random access storagemedia, media alignment marks, and digital symbols carrying size and/orstorage density indicia.

Terms such as "encoded two-dimensionally" or "two-dimensional encoding,"as used herein with respect to symbols, mean that the informationrepresented by the symbols can be determined from only thetwo-dimensional positioning of the symbols relative to either oneanother or to any other reference on the medium, and any differentiablecharacteristic intrinsic to the symbols; there is no need for any flagor flags relating to the symbols or their positions with respect to themedium itself. Examples of a differentiable characteristic intrinsic tothe symbols include optical reflectivity or transmissivity, shape,color, size, and orientation. The combination of a differentiablecharacteristic and relative position, or a locational characteristic, isreferred to herein as a differentiable-locational characteristic.

Examples of storage media capable of carrying differentiable symbolsencoded two-dimensionally include optical-storage media such as paper,discs, or film; and magnetic-storage media such as paper, tape, ordiscs.

The invention is particularly advantageous for use with practicalstorage media in which the position or location and the differentiablecharacteristics of the symbols are subject to statistical variations,i.e., they are not uniform. For optical-storage media, nonuniformity ofdifferentiable-locational characteristics of the symbols can be causedby optical and locational variations or distortions in theoptical-storage medium itself, or by inconsistencies in the applicationof the symbols to the medium. Examples of optical variations ordistortions in an optical storage medium such as motion picture filmstock include over- or under-developing, and surface defects such asscratches or smudges. Examples of locational variations in motionpicture film include stretching in one or more dimensions, andvariations in the location of symbols with respect to the film. Examplesof inconsistencies in application of symbols to the media includevariations in the position of the symbols with respect to the boundariesof an underlying medium, and variations in the density and/or shape ofthe symbols themselves.

The symbols carried by the storage medium are oversampled in twodimensions. Oversampling is sampling at a rate higher than the Nyquistsampling rate. In preferred embodiments, two adjacent symbols constitutea complete cycle of data; therefore, sampling at a rate higher than theNyquist rate means sampling at a rate higher than exactly once persymbol.

Although the invention in its broadest aspects contemplates the use ofany means for oversampling the symbols, the invention is particularlyadvantageous for use with practical sensing arrangements such as opticalsensing arrangements in which the optical characteristics and/orlocation characteristics of the symbols, referred to herein asoptical-locational characteristics, are distorted by the sensingarrangement. Such distortion can be caused by optical and locationalvariations or distortions in the optical sensing arrangement itself, orby variations in the relative location and/or motion of theoptical-storage medium with respect to the optical sensing arrangement.Examples of optical and locational variations in a sensing arrangementfor motion picture film include lenses which are out of focus or whichhave optical aberrations, and lateral and/or azimuthal misalignment ofoptical sensing elements. Examples of variations in relative locationand/or motion of a medium such as motion picture film with respect to asensing arrangement include horizontal motion or weaving within anoptical plane, horizontal motion and twisting of the film out of anoptical plane, rotational or azimuthal motion, short-term fluctuationsin film speed sometimes called jitter or flutter, and longer-termfluctuations in film speed.

A representation of the symbols carried by the storage medium is derivedfrom the samples produced by oversampling the symbols in two dimensions.This representation of the symbols carried by the storage mediumconstitutes a two-dimensional image representation. Filtering in twodimensions, in the nature of reconstruction filtering or imageenhancement, may be applied as needed to improve the resolution of thetwo-dimensional image representation so that it is suitable for locatingthe symbols and determining, within a desired accuracy, the digitalvalue of the digital information which they represent. The requirementsfor reconstruction filtering or image processing are inversely relatedto the amount of oversampling of the symbols; oversampling at asufficiently high rate reduces and may eliminate the need for imageenhancement filtering. In practical optical-storage media systems,depending on the cost and availability of electro-optical and electronicdevices, the system designer may be required to balance the amount ofoversampling and the resources required to perform the oversamplingagainst the amount of image enhancement filtering and the resourcesrequired to perform the filtering.

The digital value of the digital information represented by each symbol,referred to herein as the "symbol value," is recovered from the imagerepresentation of the symbols by examining, in an optical-storage mediasystem for example, optical-locational characteristics of the imagerepresentation. Symbol values are recovered by examining otherdifferentiable-locational characteristics if another type of storagemedia is used. Optical characteristics, or other differentiablecharacteristics in the case of non-optical storage media, are comparedto one or more references; for example, optical characteristics may becompared to light transmissivity or reflectivity thresholds. Locationalcharacteristics are compared to one or more references such as, forexample, a set of anticipated relative symbol locations.

In the preferred embodiment, the derivation of the image representationand recovery of the symbol values is done in the digital domain and maybe implemented in whole or in part using general purpose digital signalprocessing integrated circuits ("chips") and/or application-specificdigital circuitry. In principle, such derivation of the imagerepresentation and recovery of the symbol values may be done wholly orpartly in the analog domain, although probably at a greater cost due toincreased complexity.

In the preferred embodiment of the invention for motion picturesoundtrack applications, the digital information constitutes bytes ofdigital information in a binary bit stream; the bit stream represents aplurality of motion picture soundtrack channels and, optionally, otherinformation useful for the playback of a motion picture film.

The digital information represented by the symbols may itself beencoded. For example, the digital information may be an encodedrepresentation of other analog and/or digital information which has beensubject to processing such as data compression, error correctionencoding, randomizing, and formatting. The manner in which the digitalinformation is encoded does not form a part of this invention, nor doesthis invention relate to the recovery or reproduction of that which isrepresented by encoded digital information.

The invention allows the recovery of a densely packed array of smallsymbols carried by the storage medium and the determination of thedigital information represented by the symbols entirely byelectro-optical and electronic means without requiring precisepositioning techniques. The symbols carried by the storage medium needonly be within the sensing range of the sensing means. There are norequirements for close alignment between the storage medium and thesensing means nor are there requirements for close synchronization,clocking or tracking of the storage medium. No timing information or"flags" need be carried by the storage medium. In addition tosimplifying the process of information recovery, the absence of clock,timing or "flag" information makes more information carrying area of thestorage medium available for other purposes.

The image representation recovered in accordance with the presentinvention is a representation of the differentiable-locationalcharacteristics of the symbols carried by the storage medium whichallows examination of two-dimensional positional information of thesymbols.

Although the invention has many applications, it is described inconnection with the preferred embodiment wherein the storage medium isan optical-storage medium in the form of a web, namely motion picturefilm stock carrying optically recorded symbols representing digitalinformation, and recovery of the symbols and digital informationrepresented by the symbols is performed by a portion of a digital motionpicture film soundtrack playback system.

In the preferred embodiment, symbols representing digital informationare carried by the motion picture film stock in a series of discretesegments in the form of two-dimensional blocks of symbols. In principle,such segments may be of any convenient length or size. The informationis read by oversampling the blocks in two dimensions, deriving atwo-dimensional image representation of each block of symbols, anddetermining the digital value represented by each symbol in the block.

In accordance with the basic principles of the invention, there are nostringent requirements for any special alignment or synchronization ofthe film with respect to the optical sensing means, nor are thererequirements for clocking or tracking information on the film. The imagerepresentations of the blocks of symbols are two-dimensional imagerepresentations which permit the determination of digital valuesrepresented by the symbols to be made on the basis of the symbol'soptical transmissivity and location without regard to timing or "flags."

The invention operates in cooperation with conventional motion pictureprojectors and results in a system that is highly robust in the sensethat it is resistant to 1) variations in film speed, 2) film jitter, 3)film weave, 4) film azimuth errors, 5) film stretch, 6) dirt andscratches on the film, 7) variations in optical transmissivity, 8)variations in brightness of the light source for the optical sensingmeans, and 9) variations in the sensitivity of the optical sensingmeans. The system is also resistant to variations in the positions ofthe film with respect to the optical sensor by which the symbols carriedby the film are optically sensed.

In the preferred embodiment, the blocks of symbols are located betweenthe sprocket hole perforations on at least one side of the motionpicture film stock and are generally of rectangular shape. If thedigital information is placed on only one side of the film, the digitalinformation is preferably located on the side on which the analogsoundtracks are recorded. The area between motion picture film sprockethole perforations is sometimes referred to in this document as theinterperforation area or simply as an "interperf area" or "interperf. "

Each of the blocks may have one or more alignment patterns or referencesto assist locating the symbols, but the alignment patterns or referencesare not necessary. The use of alignment patterns or references may allowdetermining the location of the symbols using lower amounts ofprocessing power.

The terms "alignment pattern" and "pattern" are used herein to mean anyreference that can be used to assist aligning and/or locating theinformation-carrying symbols; they should not be understood to includeany limitation implied by the word "pattern."

If alignment patterns are employed, preferably they should haveautocorrelation properties such that low autocorrelation values resultwhen the pattern is not congruent with itself. A cross-multiplied Barkercode may be used. In the preferred embodiment, there are four alignmentpatterns, one at each corner of each block of symbols. For simplicity inprocessing as described below, the four alignment patterns areidentical; in principle, they may differ from one another. In thepreferred embodiment each of the alignment patterns is a 7-by-7 array ofsymbols and each block of symbols, including the alignment patterns, isa 76-by-76 array of symbols.

The symbols representing digital information are in the form ofoptically differentiable symbols. In the preferred embodiment, thedigital information represented by the symbols is binary. Thus, thesymbols need have only two states, such as transmissive and opaque, orreflective and non-reflective. In the preferred embodiment, the symbolsare read by sensing the amount of light they pass transmissively. Inother embodiments, the symbols may be read by sensing the amount oflight they reflect; the form in which the symbols are carried by theoptical-storage medium may be changed as necessary to facilitate sensingreflective light.

Other embodiments of the invention may use symbols with various types ofdifferentiable characteristics. For example, magnetic sensingarrangements can detect different magnetic field orientations; tactilesensing arrangements can detect different heights; electrical sensingarrangements can detect different levels of conductivity, capacitance,or inductance; and optical sensing arrangements can detect differentcolors and shapes.

In principle, the symbols could represent digital information other thanbinary information by using multiple differentiable characteristics. Forexample, each symbol may be given any of a plurality of transmissivityor reflectivity levels, colors or shapes, or some combination of suchcharacteristics.

In the preferred embodiment, each of the symbols is generally square inshape. For use in motion picture applications, the size of the symbolspreferably is sufficiently large so that high-speed motion picture filmprinting techniques can be used without encountering resolutionproblems. In the preferred embodiment, the side of each square symbol ison the order of 32 microns.

In the preferred embodiment, the symbols are located relative to oneanother in such a manner that their centers define the intersections ofan orthogonal grid having equally spaced lines. For other embodiments,the relative location of the symbols is not limited to such aconfiguration.

In the preferred embodiment, the central portion of each block ofsymbols is a 12-by-12 array of symbols which carries no soundtrack orauxiliary information. The central potion is available for other datacarrying purposes, for a further alignment pattern, or it may bedesignated as an area not to be used to carry any useful information.

In the preferred embodiment, the film also carries two conventional SVAsoundtracks. The existing analog soundtrack locations remain unalteredso that the film may carry both the conventional analog soundtrackinformation and the new digital soundtrack information. The arrangementthus provides for full compatibility with existing analog motion picturefilm formats.

Alternatively, by locating the symbols between the sprocket holeperforations on both sides of the film and/or by also locating thesymbols between the sprocket holes and the edges of the film, it may bepossible to reduce or eliminate any requirement for bit rate reduction,to provide for additional signals to enhance the system'smultidimensional sound properties, and/or provide for other purposes. Asa further alternative, if compatibility with existing motion pictureanalog soundtracks is forsaken, the symbols representing digitalinformation may be located in the area used by conventional analogsoundtracks. In that case, the symbols may be grouped into one or moreblocks of any convenient length. In addition, if desired, the symbolsmay be carried across the entire motion picture film area by knownfluorescent-optical techniques.

In a proposed commercial product embodying the present invention, thedigital information represents multiple soundtrack channels andauxiliary information. Preferably, the digital information comprisesfive 20 kHz bandwidth audio channels: left, center, right, left surroundand right surround; a subwoofer channel of about 125 Hz bandwidth; andtwo auxiliary channels of digital data, one at 2400 bits per second andthe second at 9600 bits per second. The audio and digital channels aresubject to processing including error detection/correction encoding andbit-rate reduction or data compression.

As mentioned above, the invention is applicable to applications, mediaand storage techniques other than those of the preferred embodiment. Forexample, the invention may be employed to store and recover informationfrom paper using optically or magnetically differentiable symbols byusing electrostatically applied toner or magnetic ink, respectively.Very large amounts of information such as text, music, voice, videoimages, or digital data could be stored on paper and read simply andinexpensively using ordinary noncoherent light sources.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a view of a recorded 35 mm motion picture (cinematographic)film carrying both conventional analog soundtracks and blocks of symbolscarrying digital information according to the present invention.

FIG. 2 is an expanded view of a portion of FIG. 1 in the region of thefilm sprocket holes and one of the analog soundtracks.

FIG. 3 is a view similar to that of FIG. 2.

FIG. 4 is a further expanded view of a portion of FIG. 3, showing anexemplary block of symbols representing digital information according tothe present invention.

FIG. 5 is a hypothetical graphical representation of nonuniform opticaltransmissivity characteristics of two abutting symbols carrying digitalinformation on an optical storage medium.

FIG. 6 is a functional block diagram of a motion picture film soundtrackplayback system incorporating the preferred embodiment of the presentinvention.

FIG. 7 is a schematic view of the preferred embodiment of an OpticalSensor for use with an optical storage medium in the form of motionpicture film stock.

FIG. 8 is a functional block diagram of the Video Processor portion of amotion picture film soundtrack playback system which includes thepresent invention.

FIG. 9 is a functional block diagram of the Image Processor portion of amotion picture film soundtrack playback system which includes thepresent invention.

FIG. 10 is a schematic diagram illustrating field edge search zonesassociated with a two-dimensional image representation of a field.

FIG. 11 is a hypothetical graphical representation showing thedistribution of optical samples for the symbols in a block of symbols,and a threshold by which the binary value of the information representedby each symbol may be determined.

FIG. 12 is a functional block diagram of the Audio Signal Processorportion of a motion picture film soundtrack playback system whichincludes the present invention.

MODES FOR CARRYING OUT THE INVENTION I. MEDIUM

Referring now to FIG. 1 of the drawings, the storage medium carryingsymbols representing digital information is shown in the form of amotion picture (cinematographic) film 2. Although these figures depictthe preferred embodiment of a medium in the form of a 35 mm motionpicture film which is conventional except for the addition of thesymbols representing digital information, the storage medium aspect ofthe invention is applicable to other film sizes and formats including 70mm motion picture film, and is also applicable to other media includingoptical-storage media.

FIGS. 2 and 3 show in more detail a portion of the film 2 includingsprocket holes 4, two analog SVA or dual-bilateral monophonic tracks 6and 8, and blocks of symbols 10. FIG. 3 also shows a portion of picturearea 12.

FIG. 4 shows in even greater detail a portion of the film 2 including ablock of symbols 10 between two sprocket hole perforations 4. One of theanalog tracks 8 is also shown.

A. Location and Configuration of Data Fields

The blocks of symbols comprise a 76-by-76 array of contiguous squareoptically-transmissive and opaque symbols located relative to oneanother in such a manner that the symbol centers define theintersections of an orthogonal grid having equally spaced lines.Transmissive symbols may represent a binary one or zero and opaquesymbols represent the opposite binary value. In the preferredembodiment, opaque symbols represent zeros. Throughout this document asymbol representing digital information will also be referred to as a"fixel," a shortened form of the term "film picture element."

In principle, the fixels need not be square but could be oblong,hexagonal, triangular, circular, or some other shape; the symbols neednot be contiguous; and the grid defined by the symbol centers need notbe an orthogonal grid with equally spaced lines. However, this preferredconfiguration is a good compromise offering a fairly high symbol packingdensity and a structure requiring only relatively low amounts ofprocessing to recover the information represented by the symbols.

Preferably, the optical transmissivity of each symbol is uniform acrossits entire area. It is also possible to represent binary informationwith a symbol having nonuniform optical characteristics. FIG. 5 is ahypothetical graphical illustration of the optical transmissivity ofthree abutting symbols along a line passing through their centers; thecurve for each symbol is sinusoidal. The significance of this nonuniformtransmissivity will be better appreciated by one skilled in the artafter understanding the use of equalization filtering, discussed below.

The size of each symbol is inversely related to the number of symbolswhich may be carried in a given area. In the absence of all otherchanges, digital data rates may be increased by reducing the symbolsize, but reducing the symbol size may impair the ability to accuratelydetermine the digital information represented by the symbols. Forexample, smaller symbols are more susceptible to film printing errors,corruption by the wear and tear of film usage, and focus problems in theoptical sensing arrangement. Furthermore, for optical sensingarrangements based upon transmitted light levels, the smallest usablesymbol size is constrained by diffraction effects at the edges of thesymbols.

For 35 mm motion picture film, the preferred length of each square fixelside is about 32 microns; therefore the size of a 76-by-76 array offixels (a block of symbols) is about 96 mils (2438.4 microns). Eachblock of fixels in an interperf area is located equidistantly from thetwo closest sprocket hole perforations and is horizontally centered andaligned with respect to them.

The standard horizontal width of a 35 mm motion picture film sprockethole is 110 mils; thus, the width of the interperf area transverse tothe film length is 110 mils. Two 7-mil (177.8 micron) wide opaque guardbands occupy the areas between each 96-mil wide block of fixels and theleft and right boundaries of the interperf area.

The standard vertical distance between 35 mm motion picture filmsprocket holes is 109 mils; thus, the height of the interperf area alongthe film length is 109 mils. Two 6.5-mil (165.1 micron) wide opaqueguard bands occupy the areas between each 96-mil high block of fixelsand the two adjacent sprocket hole perforations.

The location of the block of symbols is not critical to the invention,but its' placement in the interperf area does provide compatibility withmotion picture systems which utilize only the older analog soundtracks.As a result, a single inventory film with both analog and digitalsoundtracks is possible.

For 70 mm film, the general arrangement is similar because the sprockethole perforations and distances between perforations are the same.However, instead of analog soundtracks, 70 mm film employs magneticstripes carrying analog soundtrack information and those magneticstripes are located in a different place than are the 35 mm film'sanalog

A complete array of fixels in an interperf is sometimes referred to inthis document as a block of symbols or a "field."

B. Alignment Reference

In the preferred embodiment, the fixels in each block are applied to theoptical-storage medium in such a manner that the centers of the fixelsdefine the intersections of a grid of orthogonal equally spaced lines;thus, fixels have known precise physical locations relative to oneanother. It is therefore possible to establish one or more referenceswhich may assist in locating the precise center of each symbol; thereferences may be either extrinsic to the field, or they may compriseone or more symbols within the field. These references are referred toherein as "alignment patterns." Although alignment patterns are notnecessary to practice the current invention, they can greatly reduce theamount of processing required to locate the symbols.

An alignment pattern may be located at each of the four corners of eachblock of fixels. Less than four alignment patterns may be workable,depending on the particular application, the data density carried by themedium, and the processing power of the symbol recovery apparatus.Alternatively, the opaque guard bands adjacent to the edges of the blockof fixels may be used as an aid in locating the symbols and determiningtheir digital values.

As mentioned above, the central portion of each field or block of fixelsmay be reserved for other data carrying purposes, for a furtheralignment pattern, or may not carry any useful information. In thepreferred embodiment, a 12-by-12 symbol array area represents nosoundtrack or auxiliary information and is reserved for future uses. Inthe embodiment shown in the drawings, the central 12-by-12 fixel area ofthe digital information block depicts a registered trademark of DolbyLaboratories Licensing Corporation, the "Double D" symbol. This is mostclearly seen in FIG. 4.

If used, each of the one or more alignment patterns should comprise anarray of fixels having good autocorrelation characteristics. A 7-by-7array is employed in the preferred embodiment. The autocorrelationcharacteristics of the alignment pattern are such that a lowautocorrelation value results when the pattern is not congruent withitself. A cross-multiplied 7-bit Barker code (1110010₂) satisfies thedesired autocorrelation characteristics. Preferably, if more than onealignment pattern is used, all of the alignment patterns contain thesame array of fixels and are located in each corner of the block offixels. In the preferred embodiment, a band of opaque fixels is placedalong the two inside edges of the alignment pattern, the alignmentpattern and the band of opaque fixels thus forming an 8-by-8 array offixels.

C. Configuration of Bytes

The preferred embodiment groups the binary data represented by thefixels in each field into 8-bit bytes to facilitate use of errordetection/correction (EDC) codes. In order to assist the correction oferrors, the dimensions of each byte is chosen to minimize the number ofbytes affected by film surface defects such as scratches and dirt; thus,the fixels constituting a byte are arranged into as square an area aspossible. This dictates an area comprising an array of either 2-by-4fixels, or 4-by-2 fixels. Each of the 8-bit bytes of binary digitalinformation preferably are represented by fixels arranged on the film ina 2-by-4 array of fixels; two horizontal fixels transverse to the filmlength by four vertical fixels along the film length. Thus, each fieldor block of fixels has symbols representing 38 bytes of digitalinformation across the width of the field and 19 bytes of digitalinformation down the length of the field.

A 2-by-4 array is preferred over a 4-by-2 array for motion picture filmbecause scratches are more likely to be vertical along the direction offilm motion. A fewer number of bytes having a dimension of 2-by-4 fixelsare more likely to contain all of a scratch than bytes having any otherdimension.

For other applications, it may be preferable to group symbols intotwo-dimensional areas having dimensions different from that preferredfor motion picture film applications. The shape of the two-dimensionalareas should be chosen to minimize the number of areas affected byerror-causing phenomena.

The digital information, after encoding, may be randomized prior to itsapplication as symbols on the film so that the blocks of symbols willnot be likely to contain large transmissive or opaque areas. Theoccurrence of a large area of opaque or transmissive symbols wouldtherefore most likely indicate a large surface defect.

II. APPARATUS

FIG. 6 is a high-level functional block diagram of a motion picture filmsoundtrack playback system incorporating the preferred embodiment of thepresent invention. The Optical Sensor 100 optically scanstwo-dimensionally encoded information carried by an optical-storagemedium and generates video signals in response thereto. The VideoProcessor 120 controls the scanning rate of the Optical Sensor 100 andgenerates digital signals forming a two-dimensional image representationin response to the video signals. The Image Processor 140 filters thetwo-dimensional image representation to improve its resolution andgenerates digital information corresponding to the value of theoptically encoded information carried by the optical-storage medium. TheAudio Signal Processor 160 decodes the digital information intoelectrical signals for an audio presentation.

A. Optical Sensor

FIG. 7 is a schematic representation of the preferred embodiment ofOptical Sensor 100 which includes a portion of the apparatus foroversampling the symbols carried by the optical-storage medium. Theoptical properties of the blocks of transmissive and opaque fixels areread transmissively. Preferably, a diffuse light is used to illuminatethe medium; a diffuse light provides a lower contrast ratio which tendsto render scratches invisible.

A remotely located wide spectrum white light source 20, which may be a75-watt halogen incandescent lamp, for example, provides illumination toone side of the film 2 via an infrared (IR) blocking filter 22, a lightpipe 24, and a diffuser 26. The light pipe 24 may be a fiber opticcable. If the fiber optic cable is non-coherent, the cable itselfprovides a diffusion function and a separate diffuser may not benecessary. The IR blocking filter 22 is located at the lamp end of thefiber optic cable and the diffuser 26 is located at the film end of thecable. Diffuse light thus illuminates one side of the film 2 in theregion of the film sprocket hole perforations, between which the blocksof symbols representing digital information are carried. A lens system28 is located immediately on the opposite side of the film to opticallyfocus the image on an optical sensing means such as a 512 pictureelement ("pixel") linear charge-coupled-device (CCD) array 30.

A 512 pixel linear CCD array is preferred because a 256 pixel CCD arraydoes not allow a sufficiently high sampling rate, whereas a 1024 pixelCCD army results in too high a data rate for other components in thesystem. In theory, a 1024 or even larger size CCD array is desirableprovided that practical components are available to handle the higherdata rates. Another factor is that the scanning rate for a 512 elementdevice is such that a CCD and an analog-to-digital converter (ADC)designed for normal video applications can be employed, thus avoidingthe necessity to use a very high cost CCD array and ADC.

One suitable CCD array is the model CL-C3-0512 optical scannermanufactured by Dalsa Inc. of Waterloo, Ontario, Canada. A similardevice, the Dalsa Inc. model IL-C2-0512 is described in "Ultra HighSpeed CCD Image Sensors for Scanning Applications" by Brian C. Doody etal, Proceedings of the SPIE--The International Society for OpticalEngineering, vol. 1107, pp. 105-116. The models differ mainly in theheight of the photodiodes, the height being essentially the same as thewidth in the preferred CL-C3-0512 model.

Some sensors such as the above identified Dalsa Inc. CL-C3-0512 devicerequire a horizontal slit 32 to shield the CCD array from stray light.In one embodiment, a slit having a height of about 50 mils is locatedbetween the film 2 and either the diffuser or the end of the fiber opticcable if no diffuser is used. The slit height is not critical. Althougha linear CCD array is preferred because of its relatively low cost,small size and ruggedness, in principle other types of optical scannerscan be used subject to requirements for scanning rate and resolution.For example, the optical sensing means can be a two-dimensional CCDarray or an image orthicon tube. If a two-dimensional sensingarrangement is used, the motion picture film should be held relativelystationary in the sensing region, by a second Geneva pull downmechanism, for example, otherwise very high scanning rates must beemployed. Alternatively, a flying spot scanning system can be employed.The invention contemplates any means for oversampling from which atwo-dimensional image representation of each field may be derived.

Referring to FIG. 7, the film 2 is continuously transported byconventional means through the light transmission and sensing region inthe direction shown by the arrow. Linear CCD array 30 is mountedhorizontally, optically sampling along a line transverse to thedirection of film travel. For simplicity, the schematic drawing does notshow the means for transporting the film, nor does it show the means formounting the various components of the apparatus.

1. Optical Oversampling

A significant aspect of the present invention is oversampling thestorage medium in two dimensions. For the preferred embodiment in motionpicture film applications, the film is optically oversampled in thehorizontal direction by imaging each symbol or fixel onto more than oneCCD element or pixel, and the film is optically oversampled in thevertical direction by scanning the film or reading the CCD array at arate more than once per fixel as the film moves vertically relative tothe CCD array. The horizontal optical sampling rate is a function offixel size, optical magnification, and CCD array pixel size. Thevertical optical sampling rate is a function of fixel size, verticalfilm speed, and CCD line scanning rate.

Turning first to horizontal optical oversampling, each square fixel inthe preferred embodiment is approximately 32 microns along each side.Each pixel of the preferred Dalsa linear CCD array has a 13 micron widthand height, but the pixels centers are 14 microns apart. In thepreferred embodiment an f4 lens system images the pixel width down toabout 5.6 microns at the plane of the film. Thus, about five or six CCDarmy pixels read each 32 micron film fixel, providing a horizontaloptical sampling rate of approximately 5.7 optical samples per fixel. Bychanging the lens system appropriately, CCD arrays with other pixelwidths can be made to provide a similar number of pixels to read eachfixel.

In the present invention, there is no requirement for any fixedrelationship between particular fixels and CCD array pixels. In thepresent embodiment, for example, as the motion picture film istransported past the lens system, a particular horizontal fixel positionin the block of symbols need not be read by the same CCD array pixels.In actual practice, a particular fixel location will not be read by thesame CCD array pixels due to horizontal weave of the film.

Large amounts of film weave are tolerated by overscanning at the ends ofthe CCD array to allow the film to shift horizontally and still bewithin the sensing range or field of view of the optical sensing means.In the preferred embodiment, the image of each 76 fixel wide symbolblock optically projects onto only 431 of the 512 CCD array pixels. Theinstallation of the optical sensor should position the linear CCD arrayso that the fixel blocks image on the average is projected onto thecenter 431 pixels. If the optical sensor is mounted at the same locationin the projector as the analog track readout, a single 1024 pixel linearCCD array may be used to scan both the digital information fields andthe analog tracks.

"Overscanning," which is the scanning of the film beyond the borders ofthe blocks of symbols, should not be confused with "oversampling," whichis the taking of samples at a rate greater than the Nyquist samplingrate.

Although there may be a slight skewing of the lines of fixels on thefilm relative to the CCD array as a result of film motion, such skewingneed not be compensated by skewing the CCD array position. This isbecause the digital information is recovered from the film by deriving atwo-dimensional image representation, as described further below. In apractical embodiment of the present invention, there is tolerance ofsubstantial azimuth errors up to about 15 degrees. In principle, verysubstantial azimuth alignment errors may be tolerated if the recoverysystem has sufficient processing power.

Turning now to vertical optical oversampling, the vertical film speed inthe United States is 24 frames per second or approximately 450 mm persecond. In the preferred embodiment, the fixels are 32 microns alongeach edge; hence, the digital information on the film moves in thevertical direction at rate of approximately 14,300 fixels per second. Atthis speed, a nominal CCD line scanning rate of approximately 40 kHzprovides a vertical optical sampling rate of approximately 2.8 opticalsamples per fixel and optical aliasing is acceptably low. The verticaloptical sampling rate and optical aliasing are functions of theinterplay between the optical filtering effect resulting from therelative pixel to fixel apertures and the CCD line scanning rate.

In the preferred embodiment, the choice of fixel size, opticalmagnification, and CCD array pixel size results in a horizontal opticalsampling rate of approximately 5.7 optical samples per fixel. The choiceof fixel size, vertical film speed, and nominal CCD line scanning rateresults in a vertical optical sampling rate of approximately 2.8 opticalsamples per fixel. The vertical optical sampling rate need not be ashigh as the horizontal optical sampling rate because film motion in thevertical direction tends to "smear" or elongate the fixels, therebyreducing their harmonic information content.

The film 2 is continuously transported through the sensing region at orclose to the normal projection rate of 24 frames per second. Continuousmovement of the film cooperates with the scanning of the CCD array toprovide one of the two directions of scanning required by the system.Although the nominal CCD line scanning rate is approximately 40 kHz, theexact scanning rate is varied to closely track short term and long termvariations in vertical film speed. Thus, the present invention istolerant of substantially all short term and long term variations inspeed that are likely to be encountered in motion picture projectorsthat are currently in use, including older projectors. This aspect ofthe invention is described below in more detail.

2. Location of Optical Sensor

In principle, a motion picture film soundtrack playback systemincorporating an alternative embodiment of the present invention can beadapted to scan or read the blocks of fixels in the picture projectionarea of the projector, or a second Geneva-type pull-down mechanism couldbe provided so that each field or block of fixels is held essentiallymotionless briefly when read. In this case, a two-dimensional sensor isrequired. If the invention is used in other environments in which thestorage medium is not moving, a two-dimensional CCD array or othertwo-dimensional sensing device would be preferred in order to avoid anyrequirement to move the medium with respect to the sensing device.

An analog soundtrack must be read at a conventional location in themotion picture projector; the analog soundtrack is about 24 frames inadvance of the picture in a 35 mm system. The choice of the sensorlocation for digital soundtrack information is more flexible, however,because a compensating time delay may be easily implemented in thedigital domain. Thus, by recording the symbols representing digitalsoundtrack information on the film four or more frames in advance of theanalog tracks, i.e., 28 frames in advance of the picture, it is possibleto physically locate the digital soundtrack sensor anywhere from theconventional analog readout location upward in the projector. In orderto reduce cost and simplify the system, the digital information sensoris preferably at or near the location in the motion picture projectorwhere the analog tracks are read. In view of the time delay inprocessing and decoding the digital information relative to the analoginformation, which is essentially instantaneous, it is necessary for thedigital information to be slightly time advanced with respect to theanalog soundtracks if they are to be read at the same location in theprojector. Preferably, the relative timing relationship of the digitalinformation is optimized such that 1) a reproduction system may"switchover" from the digital to the analog soundtrack in the event ofexcessive errors or corruption of the digital soundtrack playback, and2) the film may be spliced without adversely affecting the digitalsoundtrack.

If the reading locations for the digital and analog soundtracks arelocated at the same place, a single linear array may be employed whichscans across both the digital and analog soundtrack regions on the filmand directs the information to respective digital and analog processingcircuitry. Alternatively, separate readout or sensing devices may beemployed if the digital and analog readouts are done at differentlocations or even if they are done at substantially the same location.It is also possible to locate the digital sensor in the so-called"penthouse" area above the projector in the region where the magneticsoundtracks of 70 mm films are read. When the system is used with 70 mmmotion picture, the digital readout is preferably located in thevicinity of the analog magnetic playback heads in the 70 mm magneticreadout projector penthouse.

If the digital information reading area is located at or near thelocation where the analog tracks are read, no additional film handlingmechanisms are required. If the digital information reading area for a35 mm film projector is located in a penthouse, however, an additionalfilm transport is required so that the film is substantially stable atthe readout location in a plane perpendicular to the direction of lighttransmission through the film. Such transport arrangements are wellknown in the art and one suitable transport system is a version of theDavis tight-loop film transport system described in the AudioCyclopedia, 2d ed., by Howard Tremaine, Howard W. Sams & Co.,Indianapolis 1969. See particularly FIGS. 18-28A at page 913 thereof andthe related textual material. A smaller flywheel is likely to berequired than is required for reading a conventional analog soundtrack.Although it is preferred that the film is substantially in a plane whilein the readout area, systems embodying the invention may be relativelytolerant of deviations of the film by providing a relatively broad depthof field of the CCD imaging optics. Other distortions resulting from anon-planar film readout may be inherently accommodated by the manner inwhich the invention recovers the symbols and determines the digitalvalues they represent.

B. Video Processor

FIG. 8 is a functional block diagram showing the interconnections of CCDarray 30 with the preferred embodiment of Video Processor 120. Theoutput of the CCD array 30, and hence the output of the Optical Sensor100, is a wide bandwidth video-like pulse amplitude modulated analogsignal. That signal is passed through the anti-aliasing low-pass filter40, and the black level of the signal is clamped to DC in the DC clamp42 after every scan in the same manner as that done by a televisionreceiver for a television signal. The CCD array must be over-clocked inorder to provide a black-level signal at the end of each scan. Theclamped video signal is passed to an 8-bit ADC 44 which electricallysamples the optical samples provided by the CCD array 30. The 8-bitsamples generated by the ADC 44 are stored in a random-access memory(RAM) 52. The scanned CCD array 30 taken with the film movement and theelectrical sampling by the ADC 44 provide in the RAM 52 an oversampleddigital signal of an image representation in two dimensions of thesymbols carried by the optical-storage medium.

The electrical sampling or resampling of the optical samples by the ADC44 is a practical requirement to transform the optical samples into aform suitable for subsequent processing in the digital domain. Theelectrical resamples from the ADC 44 express the analog optical samplinginformation as 256 discrete levels in 8-bit bytes suitable for digitalprocessing. As mentioned above, in principle, derivation of thetwo-dimensional image representation and recovery of the digital valuesof the symbols may be done wholly or partly in the analog domain,although probably at the cost of greater complexity and expense. Analogprocessing may not require any electrical resampling of the opticalsamples provided that the optical samples are represented electricallyas, for example, by the output of a CCD array.

As mentioned briefly above, the line scanning rate of the CCD array 30is varied according to variations in the vertical speed of the film 2 sothat optical scanning is tolerant of short term and long term variationsin film speed. A measure of vertical film speed is available from thevideo-like signal generated by the CCD array 30. The dominantlow-frequency component of this signal is a nominal 96 Hz signalgenerated in response to the light passed by alternating sprocketperforations, which are completely transmissive, and interperf areas,which are not as transmissive. This dominant low-frequency component isa direct measure of the current vertical speed of the film 2.

A variable or programmable clock arrangement for causing the CCD array30 scan rate and the ADC 44 clock rate to track the motion picture filmspeed is provided by a bandpass filter 46, a phase-locked loop (PLL)comparator 48, and a frequency divider 50. The bandpass filter 46 has abandwidth of about 100 Hz centered at the nominal 96 Hz interperf ratefor 35 mm film. This bandwidth and center frequency of the filter arescaled by 5/4 for 70 mm film. The PLL comparator 48 receives a nominal96 Hz signal from the filter 46 along with a 96 Hz reference signalderived from dividing the frequency of its nominal 11 MHz output by thefrequency divider 50. The nominal 11 MHz output of the PLL 48 is appliedas the clocking signal to the CCD array 30 and to the ADC 44. The CCDarray and ADC scan rates thus closely follow variations in the motionpicture projector film speed.

Although a nominal 11 MHz clock is applied to the particular Dalsa Inc.CCD array used in the preferred embodiment, the effective pixel clockrate is actually a nominal 22 MHz. This is because this particular 512element CCD array is configured as two interleaved 256 element devicesand the samples are produced simultaneously in two separate outputs. Oneoutput channel must be delayed by one half a sample time. Thus, whethera single 512 element CCD array is docked at 22 MHz or two interleaved256 element devices are clocked at 11 Mhz, the CCD line scanning rate isapproximately 40 kHz. This is a detail of the particular CCD array,however, and is not a feature or requirement of the invention. In thepreferred embodiment using the particular Dalsa Inc. CCD array, each ofthe two interleaved devices is clocked for 272 pixels to provide ablack-level signal for the DC clamp 42.

By clocking the CCD array 30 and the ADC 44 from the same programmableclock, the respective sampling functions run synchronously with eachother which tends to suppress the generation of undesirable artifactsthat may otherwise result from non-synchronous operation.

The ADC 44 converts or resamples 512 optical samples per scan linereceived from the CCD array 30 into 256 electrical samples per linescan, thereby reducing by one-half the oversampling in the horizontaldirection. Oversampling in the vertical direction is not reduced by theADC 44. Thus, the effective oversampling rate in both the horizontal andvertical directions is approximately 2.8 samples per fixel.

Electrical resampling should not degrade the two-dimensional opticaloversampling rates to the extent that the digital values represented bythe symbols cannot be determined to a desired accuracy. Ideally, the ADC44 would be operated at a higher rate; however, its output may need tobe limited in view of practical hardware limitations downstream in thesystem. For example, the digital signal processing chips used in onepractical implementation of the invention are limited in the amount ofRAM that they can address. If the ADC 44 is operated at too a high rate,the number of samples generated for each block of fixels will exceedthis limited amount of RAM.

Low oversampling rates result in two related, although distinct,problems: 1) the recovered image representation may have inadequateresolution in view of the number of samples taken, and 2) the cut-offfrequency of the anti-aliasing low-pass filter required by the lowsampling rate may be so low that intersymbol interference degrades therecovered image representation.

By operating the ADC 44 at a sampling rate of 256 samples per scan line,the anti-aliasing low-pass filter cut-off frequency is not so low thatintersymbol interference is excessive. Intersymbol interferencemanifests itself as a "smearing" or ringing of the transitions betweenfixels of opposite sense. To say that the intersymbol interference orsmearing is not excessive means that reconstruction of the imagerepresentation to a desired resolution, e.g., resolution enhancement, ispossible using only relatively short reconstruction filters.

C. Image Processor

FIG. 9 is a functional block diagram showing the preferred embodiment ofthe Image Processor 140 which improves the resolution of the imagerepresentation stored in the RAM 52 and generates digital informationcorresponding to the value of the optically encoded information carriedby the optical-storage medium. The pattern alignment 60 determines thelocation of one or more alignment patterns within the imagerepresentation and thereby reduces processing requirements byrestricting the area within the image representation which must be"upsampled" by the reconstruction filter 62. The adaptive equalization64 further filters the image representation, reducing the effects ofintersymbol interference. The adaptive threshold 66 generates binarydata corresponding to the digital information within the imagerepresentation; for the preferred embodiment of the optical-storagemedium described above, a one is generated in response to eachtransmissive symbol and a zero is generated in response to each opaquesymbol. The error detection/correction (EDC) 68 rectifies correctableerrors detected in the binary data.

For applications incorporating the preferred embodiment of motionpicture film described above, the output of the ADC 44 occurs in anuneven flow of blocks or bursts of digital data because of thearrangement of the fields in the interperf area of the motion picturefilm. The duty cycle of the bursts is approximately 50% because eachfield or block of fixels is separated from the next field by a sprocketperforation hole whose vertical dimension is roughly the same as thevertical height of the fields.

This digital data output from the ADC 44 represents analog samples ofthe symbols resulting from oversampling the symbols in two dimensions.The timing of the functions performed by the Image Processor 140 followthe timing of the Optical Sensor 100. Special system timingconsiderations resulting from the uneven flow of the digital data isdiscussed in more detail below.

In the preferred embodiment, the reconstruction filter 62 appliestwo-dimensional filters to the image representation generated by theVideo Processor 120 to improve the resolution of the two-dimensionalimage representation of the symbols carried by the optical storagemedium. The two-dimensional filtering, sometimes referred to herein as"upsampling," is in the nature of reconstruction filtering or imageenhancement.

Also in the preferred embodiment, the adaptive equalization 64 reducesintersymbol interference by filtering the upsampled image with adaptivesparse two-dimensional filters, and the adaptive threshold 66 generatesin response to the equalized image a binary representation of thefixels. The output of the adaptive equalization 64 and the adaptivethreshold 66 is used to adapt the equalization filter coefficients. Theadaptive threshold 66 adapts its threshold level in response tostatistical characteristics of the output of the adaptive equalization64. Both adaptive equalization and adaptive thresholding are describedbelow in more detail.

The processing requirements to perform reconstruction filtering forincreasing resolution and to perform equalization filtering for reducingintersymbol interference are inversely related to the amount ofoversampling of the symbols. Oversampling at a sufficiently high ratereduces and may eliminate resolution and intersymbol interferenceproblems. In practical systems, depending on the cost and availabilityof electro-optical and electronic devices, the system designer may berequired to balance the amount of oversampling against the processingpower required for symbol recovery and image enhancement.

Even if the Optical Sensor 100 and the Video Processor 120 are operatedat a higher sampling rate than described herein, it may still beadvantageous to perform some two-dimensional filtering to smooth andenhance the two-dimensional image representation. Althoughtwo-dimensional filtering is most economically performed in the digitaldomain, the filtering may be performed in the analog domain. Ifperformed in the digital domain, the two-dimensional filtering may beaccomplished in the preferred embodiment of the present invention byapplying two cross-multiplied one-dimensional filters.

More complex two-dimensional filters may be required for alternativeembodiments using a storage medium carrying symbols whose centers definenonorthogonal patterns. Such filters may be required, for example, torecover digital information from an array of symbols arrayed in ahoneycomb pattern.

In a practical embodiment, in order to provide sufficient cost-effectiveprocessing power with currently available digital signal processing(DSP) chips, several sets of RAM and DSP chips make up the memory andprocessing means. Based on currently available digital signal processinghardware, the preferred DSP chips are the Motorola model 56001 digitalsignal processor employed in a pipeline architecture. Each has aread-only memory (ROM), RAM for each of the processor chip's threeaddress spaces, and an interface to external input/output. Each of themultiple DSP/ROM/RAM combinations can be generically configured for thesystem, allowing the economy of manufacturing one type of board whileallowing the hardware to perform different functions by changing onlythe software in ROM. Despite practical considerations which require aseparate set of RAM chips for each DSP chip, the RAM 52 discussed hereinand depicted in the drawings refers to these RAM chips collectively.

1. Pattern Alignment

In the preferred embodiment of the present invention, the centers of thefixels define the intersections of an orthogonal grid of equally spacedlines; thus, by determining the location of two or more fixels within afield, a set of positional references can be established which cangreatly reduce the amount of processing required to determine thelocation of all other fixels within that field. Although alignmentpatterns are not required to practice the present invention, thepreferred embodiment utilizes alignment patterns in each corner of eachfixel block to reduce the amount of processing required to determine thepositional references. By determining the precise location of eachalignment pattern in the two-dimensional image representation, theexpected location of each fixel center in the image representation canbe established fairly precisely. Once the fixel centers in the imagerepresentation are determined, the light levels transmitted by thefixels at their centers are compared against references or thresholds todetermine if a binary one or a binary zero is represented by each fixelin the image representation. In performing these functions, the ImageProcessor 140 acts on the two-dimensional image representation of thefields generated by the Video Processor 120 in response to the signalreceived from the Optical Sensor 100.

Thus, in the preferred embodiment, the Image Processor 140 generates abinary representation of the information carried by the fixels bydetermining the position of the fixels within the image representation,examining the value of light transmitted by each fixel to determinewhether a fixel is transmissive or opaque, and generating a one or zeroin response to a transmissive fixel or an opaque fixel, respectively.

More specifically, in the preferred embodiment represented in FIG. 9,the pattern alignment 60 determines the position of the fixel centers byfirst locating one or more alignment patterns relative to which fixelpositions can be established. The pattern alignment 60 looks for thealignment patterns in the fixel block image representation. As mentionedabove, the alignment patterns are based on the 7-bit Barker code. Barkercodes are a class of bit patterns well known in the art which have goodautocorrelation properties. The Barker codes have the sameautocorrelation peak as any other bit pattern, a value equal to numberof bits the code is wide, but their autocorrelation side lobes are onlyone unit high.

The 7-by-7 alignment pattern is a cross-multiplication of the 7-bitBarker code. The alignment pattern in the image representation, aftertwo-dimensional oversampling, comprises a 20-by-20 array of 400 samplesin the RAM 52. The expected values of the samples in an alignmentpattern, referred to herein as a "known pattern," comprise a 20-by-20array of 400 elements. A cross-correlation score between samples in theimage representation and a known pattern is obtained bycross-multiplying 400 samples in the image representation by 400elements in the known pattern. The cross-correlation peak is found byshifting the known pattern in one dimension by one sample relative tothe image representation, cross-multiplying to determine across-correlation score, shifting, and cross-multiplying until a peakcross-correlation score is obtained. The position of the known patternwhich yields a cross-correlation peak establishes with a precision ofone sample the position of the alignment pattern within the imagerepresentation.

It is desirable, however, to determine the position of the alignmentpattern with even greater precision. A four-fold increase in resolutionis possible by defining a set of sixteen distinct known patterns, eachof which represents the expected values of the samples in an alignmentpattern shifted by increments of one-quarter of the interval betweensamples in either or both of two dimensions, and by determining which ofthe sixteen known patterns provides the largest cross-correlation peakwith the image representation. Each of the sixteen known patternsrepresent one of four phase shifts in horizontal sampling and one offour phase shifts in vertical sampling. The set of sixteen quarter-phaseknown patterns comprise an 80-by-80 array of 6400 elements which can bepre-computed and stored in ROM, or computed and stored in RAM as needed.

If sufficient processing power is available in the digital domain,cross-correlation scores may be obtained between the entire imagerepresentation and the known correlation patterns to determine thealignment pattern locations. The amount of processing power required tolocate alignment patterns may be reduced by either estimating thelocation of each pattern and beginning the search at the estimatedlocation, or by confining the search to areas within the imagerepresentation in which the patterns are most likely to be found.

The pattern alignment 60 comprises an edge detector which estimates theposition of each alignment pattern by finding the edges of each fieldwithin a precision of one fixel. The edge detector is essentially likean alternating-current detector, examining a line of samples within theimage representation and determining the uniformity of the samples. Ifthe samples are not uniform, the edge detector concludes that the lineof samples lies within the field. If the samples are essentiallyuniform, the edge detector concludes that the line of samples lieswithin one of the opaque guard bands surrounding the field. See FIG. 4.

In the preferred embodiment, the edge detector confines its search to arectangular region within the image representation referred to herein asa "search zone." A distinct search zone of 50-by-120 samples is definedfor each of the four field edges such that the zone abuts the boundaryof the image representation adjacent to its respective field edge. The"exterior border" of each search zone abuts a boundary of the imagerepresentation. The "interior border" of each search zone is on theopposite side of the zone from its exterior border. The longer dimensionof each search zone is parallel to its exterior and interior borders,and parallel to its respective field edge. FIG. 10 illustrates the fieldedge search zones. Under ideal conditions, the field 210 issubstantially centered in the image representation 200 which is storedin the RAM 52, and each field edge is approximately centered between theexterior and interior borders of its respective search zone. Underpractical conditions, film weave and jitter cause the field to movearound in the image representation; however, each search zone is largeenough to insure that its respective field edge falls between itsexterior and interior boundaries under nearly all conditions.

For example, referring to FIG. 10, the search zone 220 for the left-handfield edge 212 is a region whose left-hand border 222 abuts theleft-hand boundary 202 of the image representation 200, and whose upperborder 226 and lower border 228 are equidistant from the upper boundary206 and the lower boundary 208 of the image representation 200. Underideal conditions, the field edge 212 is approximately centered betweenthe exterior and interior search zone borders 222 and 224, respectively.

In the preferred embodiment of the present invention, the edge detectorof the pattern alignment 60 starts its search for each field edge at theinterior border of the respective search zone. The edge detectormeasures the uniformity of a set of 39 samples spaced 3 samples apartlying along the interior border. If the samples are not uniform, thenthe edge detector examines another set of 39 samples lying along a linefive samples closer to the exterior border and measures theiruniformity. The edge detector reiterates these steps until the samplesin a set are sufficiently uniform, thereby indicating the set of sampleslie along a line that is within the opaque guard band adjacent to thefield edge being sought. Having found a line of uniform samples in theguard band, the edge detector moves back toward the interior border andselects a point 2 samples away from the line of uniform samples; thispoint is the estimated location of the field edge. The edge detectorrepeats these steps for each edge.

The edge detector measures sample uniformity by summing the absolutevalue of the difference between adjacent samples within the line of 39samples and comparing the sum to a threshold. For implementations inwhich the value for each sample is substantially in the range from -0.5to +0.5, the threshold is 0.125, an experimentally determined value. Ifthe sum exceeds the threshold, the samples are not uniform and the edgedetector continues the search. If the sum does not exceed the threshold,the samples are sufficiently uniform for the edge detector to concludethey lie within one of the opaque guard bands.

If a field edge is not found before ten iterations are completed, thesearch for the edge is abandoned and the location of the correspondingedge which was most recently found in a previous field is used instead.

Having found the edges, the location of each alignment pattern can beestimated. The cross-correlation search for each alignment patternbegins at the estimated location, which is the intersection of the twoedges adjacent to the respective pattern.

The next step performed by the pattern alignment 60 is to locate eachalignment pattern within a precision of one sample. This is accomplishedby finding the peak cross-correlation score between the imagerepresentation and a seventeenth known pattern derived from an averageof four central known patterns from the set of sixteen quarter-phaseknown patterns discussed above. The central four patterns may be denotedas patterns KP(1,1), KP(1,2), KP(2,1), and KP(2,2), where {KP(i,j)}represents the set of sixteen quarter-phase known patterns, where 0≦i<4and 0≦j<4, and where i and j each representing in quarter-phaseincrements the amount of phase shift in one of two dimensions.

The cross correlation between the image representation and theseventeenth known pattern has a broader peak than the cross correlationsbetween the image representation and any of the quarter-phase knownpatterns; therefore, the initial search in the image representation foreach alignment pattern is assumed to begin on the skirts of this broadercorrelation peak. This assumption permits the pattern alignment 60 todirect its search based upon the gradient of two or more trialcross-correlation scores. The search is directed "uphill" along thecorrelation peak toward the peak score. This search is more efficientthan an exhaustive search in a particular area of the imagerepresentation.

In practice, the maximum theoretically possible cross-correlation scoremay not be found because of a failure to accurately recover all of thealignment pattern symbols, or because of a deviation from expectedsample values caused by any of several optical variations. In suchcases, the peak cross-correlation score may still be useful even thoughit does not equal a theoretical maximum score, which in the preferredembodiment, is approximately 0.8 to 0.9. The preferred embodiment of thepresent invention, therefore, maintains an exponentially decayingaverage of the peak cross-correlation scores achieved for the fields andestablishes a threshold at some fraction of the exponentially decayingaverage. Cross-correlation scores in excess of the threshold are assumedto indicate an alignment pattern has been found.

In the preferred embodiment, limited processing power dictates that theexponentially decaying average be recalculated for every second field,according to

    A.sub.m =(X-A.sub.m-1)·E+A.sub.m-1                (1)

where

A_(m) =average score of previous fields up to and including field m,

E=factor establishing the rate of decay, and

X=cross-correlation peak for field m.

The preferred embodiment uses a decay factor E of 0.01, providing a rateof decay of 1% across two fields. The threshold is established at 67% ofthe exponentially decaying average, although a minimum level is alsoestablished below which the threshold may not fall. This minimum value,which in the preferred embodiment is equal to 0.2, helps assure thatsome arbitrary pattern will not be mistakenly identified as an alignmentpattern during intervals when a failure to locate any pattern wouldotherwise cause the exponentially decaying average to drop to a very lowlevel.

If the peak correlation score exceeds the threshold, the patternalignment 60 assumes that it has located an alignment pattern within aprecision of one sample. It then refines the expected location to withina precision of one-quarter sample as described above by identifyingwhich of the sixteen quarter-phase known patterns provides the largestpeak cross-correlation score.

After the alignment patterns in each of the four corners of a field arelocated, a grid of positional references may be defined which coincidewith the centers all fixels within the field. In the preferredembodiment, the grid nominally comprises the intersections of equallyspaced orthogonal lines. In practical embodiments, aberrations in thestorage medium such as warping or stretching can distort the grid. Byusing four alignment patterns, one pattern in each of four corners of afield, and linearly interpolating the position of the grid lines betweenthe alignment patterns, the preferred embodiment of the presentinvention can accommodate any linear distortion. Nonlinear forms such aspincushion or barrel distortion may be accommodated by using additionalalignment patterns to segment the distorted field into regions whosedistortion may be approximated as a linear distortion.

If the pattern alignment 60 cannot locate a particular alignmentpattern, the location of the missing alignment pattern may be estimatedrelative to the position of other alignment patterns which are locatedwithin a given field. The location of a missing alignment pattern mayalso be estimated from the location determined for the correspondingalignment pattern in a prior field.

In the preferred embodiment, if the pattern alignment 60 fails to find aparticular alignment pattern, it estimates the location of thesought-for alignment pattern by using the location of an adjacentpattern found in the current field and the distance between thesought-for pattern and the adjacent pattern in a previous field. Thepattern which is adjacent in the horizontal dimension is used to makethe estimate in preference to the pattern adjacent in the verticaldimension. Using the estimated location, the pattern alignment 60attempts again to locate the pattern. If the second attempt also failsto locate the alignment pattern, the search is abandoned and theestimated location is used instead.

For example, if the alignment pattern in the upper-left corner of thefield, referred to here as pattern UL, is not found but the upper-rightpattern UR is found, then a search for pattern UL is conducted againusing an estimated location derived from the location of pattern UR andthe distance between the last found patterns UL and UR in a previousfield. If pattern UL is found but pattern UR is not found, then thelocation of pattern UR is estimated relative to pattern UL.

If patterns UL and UR are not found but pattern LL is found, then thelocation of pattern UL is estimated relative to pattern LL. If patternsUL, UR, and LL are not found but pattern LR is found, the locations forall other patterns are estimated relative to pattern LR. If no patternsare found in the current field, the last found positions of patterns ina previous field are used.

The logic used for additional combinations of missing and found patternswill be apparent to one skilled in the art.

Although the alignment pattern search may be conducted entirely in thedigital domain, some or all of the search may be performed by analogcircuitry applied to the video signal generated by the CCD array 30prior to the conversion into the digital domain by the ADC 44. Forexample, such circuitry could examine a sequence of 20 samples from theCCD array 30 and generate a signal when a sufficient number of samples,say 18, match an expected pattern. This signal can be used, for example,to interrupt the Video Processor 120 so that it may record the RAMaddress where the current digital sample is being stored. This RAMaddress establishes an estimate where the alignment pattern is likely tobe, and provides a starting point for the autocorrelation-based searchdescribed above. Other variations may be used without departing from thepresent invention.

2. Reconstruction Filter

After the position of one or more alignment patterns has beendetermined, the reconstruction filter 62 increases the resolution of theimage representation in the neighborhood of each fixel by applying atwo-dimensional interpolating or reconstruction filter to the image ateach expected fixel center. It is desirable to determine whether a fixelis transmissive or opaque by examining the amount of light transmittedat the fixel center. Because of the low oversampling rate, in general nosample will have been taken at the fixel center. By using reconstructionfilters to increase the resolution of the image, however, it is possibleto predict what a sample at a fixel center would have been had itactually been taken.

This prediction is accomplished by a two-dimensional reconstructionfilter which upsamples the image representation. A virtual times-fourupsampling of the samples is achieved by interpolating a value from thesamples in the image representation which are nearest the expected fixelcenter. Virtual upsampling provides samples close to where the fixel iscentered rather than wherever the samples happened to have been taken.

This virtual times-four upsampling increases the effective sampling rateto 1024 samples per interperf area in both the horizontal and verticaldimensions. The effect of such upsampling can be viewed as imageenhancement which yields an image representation of sufficientresolution to provide a sample no farther away from a fixel center thanabout 8 to 10% of the fixel size. Empirical results show that thisresolution is sufficient to accurately recover digital information undersubstantially all film conditions such as film weave, warpage, and minorsurface defects caused by normal wear.

Alternatively, embodiments using other hardware components may be ableto avoid reconstruction filtering by oversampling at a rate of 1024samples per interperf area in both the horizontal and verticaldimensions rather than at only 256 samples per interperf area. As notedabove, however, it still may be desirable to provide some filtering inorder to smooth the resulting image.

The preferred embodiment does not filter all samples in the imagerepresentation; it reduces the amount of processing required byfiltering only those samples near an expected fixel center. Theinterpolation or upsampling for each fixel is achieved by selecting onefilter from a set of sixteen 3-by-3 two-dimensional reconstructionfilters, and applying the selected filter to the nine samples nearesteach expected fixel center. The choice of which filter to use from theset of sixteen depends upon the offset between the expected fixel centerand the nearest sample in the image representation.

Each of the sixteen 3-by-3 two-dimensional reconstruction filters is anonsymmetrical filter with a builtin phase shift for either or both oftwo dimensions in increments of one-quarter of the interval betweensamples. A reconstruction filter RF(i,j) is constructed bycross-multiplying two three-point one-dimensional filters F(i)·F(j),where 0≦i<4 and 0≦j<4, and where i and j each represent the amount ofquarter-sample shift in one of two dimensions. These filters are veryanalogous to the sixteen quarter-phase known patterns discussed above.Each of the one-dimensional filters is derived from a symmetrictwelve-point FIR filter, the coefficients of which are shown in Table I.

                  TABLE I                                                         ______________________________________                                        Filter Coefficients for Twelve-Point FIR Filter                               ______________________________________                                        a.sub.0 = -.0682344                                                                         a.sub.4 = 0.689112                                                                         a.sub.8 = 0.459192                                 a.sub.1 = 0.02907564                                                                        a.sub.5 = 0.829184                                                                         a.sub.9 = 0.2162036                                a.sub.2 = 0.2162036                                                                         a.sub.6 = 0.829184                                                                         a.sub.10 = 0.02907564                              a.sub.3 = 0.459192                                                                          a.sub.7 = 0.689112                                                                         a.sub.11 = -.0682344                               ______________________________________                                         Filter F(0) comprises coefficients a.sub.0, a.sub.4, and a.sub.8 ; filter     F(1) comprises coefficents a.sub.1, a.sub.5, and a.sub.9 ; filter F(2)        comprises coefficients a.sub.2, a.sub.6, and a.sub.10 ; and filter F(3)       comprises coefficients a.sub.3, a.sub.7, and a.sub.11 .                  

3. Adaptive Equalization

As discussed above, accurate determination of whether a fixel istransmissive or opaque may be hindered by intersymbol interference. Theadaptive equalization 64 performs additional digital filtering, referredto herein as equalization, which may reduce and ideally may eliminateintersymbol interference. In the preferred embodiment of the presentinvention as shown in FIG. 9, equalization adapts in response to theoutput of both the adaptive equalization 64 and the adaptive threshold66.

Conceptually, an array of upsampled data at the fixel or symbol centersis filtered by an equalization filter to minimize intersymbolinterference. The equalizer is implemented by a two-dimensional FIRfilter represented by ##EQU1## where C(i,j)=equalizer filter coefficientfor tap (i,j),

D_(m) (x,y)=upsampled data for symbol at position (x,y) in field m,

Q_(m) (x,y)=equalizer output for symbol at position (x,y) in field m,

Lx=length of equalizer filter (number of taps) in the x dimension, and

Ly=length of equalizer filter (number of taps) in the y dimension.

In an embodiment of the present invention in which the digitalinformation is represented by an orthogonal array of symbols, a basicequalization filter may be implemented by a sparse 5-tap two-dimensionalFIR digital filter composing a center tap for the recovered symbol andfour adjacent taps, one tap for each of the four immediately adjacentsymbols. The filter coefficient C(0,0) corresponds to the center tap.

In one embodiment of the present invention described above for recoveryof digital information from motion picture film stock using an equalizerwith fixed-value coefficients, each filter-tap coefficient other thanC(0,0) may assume a value within a range substantially from 0.1 to 0.3,in which a value of 0.135 is generally optimum. The value forcoefficient C(0,0) is 1.

In the preferred embodiment of the present invention, however, anadaptive equalizer optimizes the accuracy of the recovered digitalinformation by adjusting its filter-tap coefficients to minimizeintersymbol interference. The amount of intersymbol interference ismeasured by the cross-correlation scores of the equalizer output of eachfield with a thresholded representation of the field.

The threshold representation is obtained according to ##EQU2## whereT_(m) (x,y)=thresholded representation of symbol at position (x,y) infield m, and

TH=threshold for determining the thresholded representation.

The establishment of threshold TH is discussed below in more detail.

The cross-correlation score of the equalizer output and the thresholdedrepresentation used to measure the amount of intersymbol interference isestablished according to ##EQU3## where Nx=number of symbols in a fieldin the x dimension,

Ny=number of symbols in a field in the y dimension, and

S_(m) (h,v)=cross-correlation score at offset (h,v) for field m.

In an embodiment of the adaptive equalizer implemented by the 5-tap FIRfilter discussed above, each filter-tap coefficient dominates only onecross-correlation score. Hence, the number of useful cross-correlationscores equals the number of filter taps minus one.

It is possible to fix the value of coefficient C(0,0) at one. Theremaining coefficients are adjusted adaptively according tocross-correlation scores. A cross-correlation score taken to one side ofthe current sample, for example, measures the amount by which thecurrent symbol smears to that side. It is assumed that the symbol on theopposite side of the current symbol smears into the current symbol bythe same amount, therefore the cross-correlation score on one side isused to adjust the FIR filter tap coefficient on the opposite side. Ingeneral, the cross-correlation score of equalizer filter tap (x,y) isused to adjust the coefficient corresponding to filter tap (-x,-y), thatis, the filter tap mirrored about the center tap from thecross-correlation score tap.

If the array of symbols carrying the digital information in each fieldis known to have good autocorrelation properties, that is, a very lowautocorrelation score for all offsets except (0,0), then intersymbolinterference can be minimized by adapting the equalizer filtercoefficients until the cross-correlation scores of the equalizer outputwith the thresholded data are zero. A basic implementation of adaptiveequalization is discussed first under the assumption that each field hasvery good autocorrelation properties. After this basic implementation isintroduced, a more general implementation of the adaptive equalizationfilter is discussed.

If sufficient processing power is available, it is possible to calculateall cross-correlation scores and adjust all equalization filter-tapcoefficients for each field of symbols. In one practical implementationof the present invention, however, only enough processing power isavailable to calculate one cross-correlation score for every secondfield. It therefore becomes necessary to utilize the cross-correlationscores for a series of fields. Presumably, each field experiencessimilar amounts of image degradation; hence, it should be possible toadapt filter-tap coefficients and converge to an optimal solution bycalculating one cross-correlation score for every second field.

Adjustment of filter-tap coefficients is accomplished by subtracting afraction of each normalized cross-correlation score from thecorresponding equalizer filter tap. Each cross-correlation score isnormalized by dividing it by the score S(0,0). Normalization eliminatesdependence upon various factors such the range of values in theequalizer output representing symbols, and the number of symbolscomprising a field. Thus, filter-tap coefficients are adjusted accordingto ##EQU4## where f=convergence factor.

The value for the factor f must be large enough to provide sufficientlyrapid convergence, yet not too large to cause instability in convergencedue to variations in the upsampled data amplitudes encountered betweenfields of symbols. Empirical evidence suggests the factor f may assumeany value within the range between 0.002 and 0.02 in the preferredembodiment. The preferred value is 0.0625.

Convergence is achieved for a particular equalizer filter-tapcoefficient when its corresponding correlation score is zero, thusindicating that smearing or intersymbol interference is cancelled.

In general, contrary to the assumption made above for the sake ofdiscussion, each field does not necessarily have good autocorrelationproperties. Adaptive equalization can compensate by determining theactual autocorrelation properties of each field. In implementationswhich do not have sufficient processing power to perform suchcalculations for each field, it becomes necessary to calculate theautocorrelation scores for a series of fields. Presumably, each fieldpossesses similar characteristics; hence, the preferred embodiment ofthe present invention establishes an average set of autocorrelationscores by calculating one autocorrelation score for every second field.

Autocorrelation characteristics may be determined from the thresholdedrepresentation of the digital information according to ##EQU5##

Because of the special properties of the T matrix, i.e., the absolutevalue of all T matrix elements is one, the autocorrelation value M(0,0)is constant and equal to the product Nx·Ny. In an embodiment of thepresent invention with a field comprising a 76-by-76 array of symbols,M(0,0)=76·76=5776.

By using normalized autocorrelation scores, equalization filter-tapcoefficients may be adjusted without regard for variations inautocorrelation properties including the size of the array of symbols,that is, the value of M(0,0). Coefficients are adjusted according to##EQU6##

It should be appreciated that implementations of the present inventionusing limited processing power cannot calculate autocorrelation andcross-correlation for every field.

An alternative embodiment of an optical-storage medium which carriessymbols with nonuniform density may reduce or eliminate intersymbolinterference, thereby reducing or eliminating the need for equalizationfiltering. For example, FIG. 5 is a hypothetical graphical illustrationof the optical transmissivity of three abutting symbols A, B, and C,along a line which passes through their centers. Symbol A is opaque, andsymbols B and C are transmissive. The transmissivity characteristics ofeach symbol is nonuniform, varying as a raised sinusoid which is afunction of distance from the symbol center.

Symbols such as those represented in FIG. 5 require less equalizationfiltering because the transitions between such symbols of opposite typehave greatly reduced high-order harmonics. Transitions between uniformsymbols, on the other hand, possess much more significant high-orderharmonics.

4. Adaptive Threshold

After the image representation in the RAM 52 has been enhanced by thereconstruction filter 62 and the adaptive equalization 64, the adaptivethreshold 66 compares the amount of light transmitted at or near thefixel centers with a threshold value to determine whether a fixel istransmissive or opaque. In effect, the adaptive threshold 66 translates8-bit values representing shades of grey within the image representationat the fixel centers into a binary representation. This is done byselecting one or more thresholds and applying them against the 8-bitvalues. Such thresholds may be dynamically altered to track thetransmissivity of the film, the brightness of the illumination, densityof the symbols, and other optical variations.

In the preferred embodiment of the present invention, an automatic gaincontrol (AGC) normalizes the 8-bit values so that zero corresponds toopacity or no transmissivity, and 255 corresponds to transparency orcomplete transmissivity. The low end of the range is established by theDC clamp 42 which clamps the "black-level" output of the CCD 30 to DC.The high end of the range is established by means not shown in FIG. 8which adjust the reference voltage of the 8-bit ADC 44 to achieve adigital output of 255 during intervals when the CCD 30 is receivinglight through the sprocket perforations. An alternate form of AGC may beimplemented digitally by scaling the output of the ADC 44, particularlyif an ADC with a resolution greater than 8 bits is used.

In one embodiment of the present invention described above for recoveryof digital information from motion picture film stock, equalizer outputfor each fixel falls within a range substantially from -0.5, whichrepresents an opaque fixel, to +0.5, which represents a transmissivefixel. Values at the equalizer output representing the fixels ideallyshould cluster tightly around these two values, but because the opticalcharacteristics of transmissive fixels are generally corrupted more thanthose of opaque fixels by effects such as film wear and dirt, the samplevalues of opaque fixels tend to cluster more tightly than those fortransmissive fixels. As a result, for embodiments of the invention usinga fixed-value threshold, the value for threshold TH is preferablyselected somewhere between the mid-point of the representation range andthe end of that range representing opaque fixels.

FIG. 11 illustrates a hypothetical distribution of the symbol valuesrepresenting the fixels at the equalizer output, referred to here as the"fixel values." The fixel value 310 is the mean value for fixels whichare opaque. The fixel value 320 is the mean value for fixels which aretransmissive or clear. The threshold is represented by the line 300.

Empirical evidence has shown that a fixed-value threshold within a rangesubstantially from -0.2 to 0.0 gives good results. A threshold equal to-0.1 is generally optimum. The preferred embodiment of the presentinvention, however, uses an adaptive-value threshold described below inmore detail.

In yet another embodiment, the determination of whether a fixel istransmissive or opaque may be accomplished by comparing the 8-bit valuesrepresenting each fixel against two distinct thresholds; those valueswhich exceed threshold a first threshold T1 are classified asrepresenting a binary one, those values which fall below threshold asecond threshold T2, where T2 is lower than T1, are classified asrepresenting a binary zero, and those values which fall in betweenthresholds T1 and T2 are classified as an error or "erasure." Erasureinformation may be used advantageously by the EDC 68 discussed below.

When a foreign particle such as dirt covers up some fixels, those fixelsappear to be opaque, all representing binary zeros. When scrapes ordefects in the film development process occur, fixels within the scrapeor defect appear to be transmissive, all representing binary ones.Because the data recorded on the film is preferably randomized, it isunlikely that large opaque or transmissive areas will occur other thanwhen a foreign particle or other defect corrupts the digital filminformation. Accordingly, the thresholding function may recognize theseareas as uncertain values rather than as zeros or ones.

In the preferred embodiment of the present invention, an adaptivethreshold is established in response to the output of the adaptiveequalization 64. The adaptive threshold 66 calculates a histogram offixel values. If sufficient processing power is available, it ispossible to calculate the histogram for each field of symbols. In onepractical implementation, however, the histogram is calculated in twopieces; an exponentially decaying average similar to that shown above inequation 1 is calculated for each of two portions of the histogram. Morespecifically, the average for each portion is calculated every fourthfield, and the decay factor E is 0.08.

After establishing the histogram, the adaptive threshold 66 selects anintermediate fixel value, say zero, finds the histogram peak below theintermediate value to determine an "opaque fixel value," and finds thehistogram peak above the intermediate value to determine a "clear fixelvalue." Starting at the opaque fixel value, the adaptive threshold 66searches up to the clear fixel value to find a "first minimum" fixelvalue corresponding to the minimum histogram level. If the minimumhistogram level occurs for more than one fixel value, the highest ofsuch fixel values is used. The adaptive threshold 66 then starts at theclear fixel value and searches down to the opaque fixel value to find a"second minimum" fixel value corresponding to the minimum histogramlevel. If the minimum histogram level occurs for more than one fixelvalue, the lowest of such fixel values is used. Finally, the adaptivethreshold 66 calculates a decaying average similar to that shown abovein equation 1 for the threshold TH using the average of the firstminimum and the second minimum fixel values as the X value, and a decayfactor E of 0.08.

The adaptive threshold 66 generates a binary value for each symbol inthe image representation and stores the binary values in the RAM 52. TheRAM 52 is not shown in FIG. 9. A one is generated for each symbol valuewhich exceeds the threshold, and a zero is generated for each symbolvalue which does not exceed the threshold.

5. Error Detection/Correction

In the preferred embodiment of the present invention, the EDC 68 isimplemented in a fairly conventional manner as a microcode controlledstate machine which drives a Reed-Solomon error detection/correctionchip number AHA4010-01 manufactured by Advanced Hardware Architectures(AHA) of Moscow, Idaho, United States of America. In principle, the EDC68 can be implemented with a very fast general purpose digitalprocessor, but such processors are expensive. A state machinearchitecture provides an economical implementation with sufficientprocessing speed.

In the preferred embodiment, the binary information received from theadaptive threshold 66 contains two levels of protection. The first levelof protection is provided by "inner" EDC codes. The second level ofprotection is provided by "outer" EDC codes. The EDC 68 passes thebinary data stored in the RAM 52 to the AHA chip for processing of theouter EDC codes, stores the results in the RAM 52, subsequently passesthese results through the AHA chip a second time to process the "inner"EDC codes, and stores the "corrected information" in the RAM 52.

D. Audio Signal Processor

FIG. 12 is a functional block diagram showing a preferred embodiment ofthe Audio Signal Processor 160. The fixed delay 70 delays the correctedbinary data to provide for installation-dependent adjustments tosynchronize playback of the digital soundtrack with the picture portionof motion picture film. These adjustments are required by variations inacoustic delays of different motion picture theaters, and by variationsin the location of the optical sensor on the motion picture projectorrelative to the lens. The first-in-first-out (FIFO) buffer 72 receivescorrected binary data from the fixed delay 70 and stores it. The erroranalysis 74 analyzes the corrected binary data stored in the FIFO buffer72 to determine whether it contains any uncorrected errors, thus makingthe information unsuitable for motion picture film soundtrack playback.As a result of the analysis, each block of data corresponding to a fieldof fixels is marked as either a "good" data block or a "bad" data block.The audio decoder 80 receives blocks of corrected binary data along withan indication whether each block is good or bad, and decodes the binaryinformation into a series of digital signals suitable for generating ananalog audio signal. The level monitor 76 tracks the amount of datacurrently stored in the FIFO buffer 72 and adjusts the operating speedof the audio decoder 80 to conform to the average rate at which thebinary data is received by the buffer. The switch over 78 switches tothe conventional analog soundtracks whenever the error analysis 76detects too many bad blocks.

1. Error Analysis and Switch Over

The error analysis 74 determines whether the binary information storedin the FIFO buffer 72 contains any errors which could not be correctedby the EDC 68. If uncorrectable errors are present, the error analysis74 marks the entire block of binary information as a bad data block.This indication is passed along with the data to the audio decoder 80,and it is passed to the switch over 78.

If too many bad data blocks are encountered, the switch over 78 selectsan alternate signal for film soundtrack playback by switching from theoutput of the audio decoder 80 to the signal obtained from theconventional analog soundtrack. In one embodiment of the presentinvention, a counter is incremented from zero up to seven for each baddata block encountered, and that counter is decremented down to zero foreach good data block encountered. A switch over to the analog soundtrackoccurs while the counter has a value of three or more. Many other switchover schemes are possible without departing from the scope of thepresent invention.

2. Audio Decoder

The audio decoder 80 comprises a decoding means and means such adigital-to-analog converter (DAC) for generating one or more analogsignals in response to good data blocks received from the FIFO buffer72. If a bad data block is received, the audio decoder 80 attempts toconceal the error by repeating the last received good data block. If toomany bad data blocks are received, the audio decoder 80 mutes itsoutput. In the preferred embodiment of the present invention, the audiodecoder 80 increments a counter from zero up to seven for each bad datablock encountered, and decrements that counter down to zero for eachgood data block encountered. The last good data block is repeated whilethe counter has a value from one to three, and the output is muted forvalues greater than three.

Many other muting and error concealing schemes are possible withoutdeparting from the present invention. Preferably, the output of theaudio decoder 80 mutes simultaneously with or immediately prior to theswitch over to the analog soundtrack by the switch over 78.

Details of implementation for the decoding means of the audio decoder 80are beyond the scope of the present invention.

3. FIFO Buffer Level Monitor

The level monitor 76 regulates the operating speed of the audio decoder80 to conform to the avenge rate at which corrected binary data isstored in the FIFO buffer 72. If the avenge operating speed of the audiodecoder 80 is too low, the FIFO buffer 72 will overflow. If the averageoperating rate of the audio decoder 80 is too high, the FIFO buffer 72will be unable to provide the decoder with the information it requiresto continue generating audio signals.

The rate at which binary data is stored into the FIFO buffer 72 is veryuneven because the output of the ADC 44 occurs in bursts of digital datawith a duty cycle of approximately 50%; however, the audio decoder 80must operate at a very smooth rate to avoid generating objectionableaudible artifacts. The level monitor 76 adjusts the operating speed ofthe audio decoder 80 by generating a clock signal whose frequency is afunction of the fullness level of the FIFO buffer 72. Preferably, thelevel monitor 76 can provide a range of from approximately -7% toapproximately +11% of the nominal rate required to decode fields scannedat a rate of 96 Hz.

In the preferred embodiment of the present invention, the level monitor76 is a variable frequency synthesizer comprising a high-frequency PLLcircuit with a crystal reference for frequency stability, and a variablefrequency divider. An example of a device providing such a function ischip number DP8531 manufactured by National Semiconductor Corp. of SantaClara, Calif., United States of America. As the fullness of the FIFObuffer 72 increases, the divisor of the frequency divider is decreased,thereby increasing the frequency of the clocking signal for the audiodecoder 80. As the fullness of the FIFO buffer decreases, the divisor isincreased to reduce the clocking frequency. The clock signal controlsthe operating speed of the audio decoder 80 by driving one or more DACwhich request decoded digital data from the decoding means. The decodingmeans in turn request information from the FIFO buffer 72 by way of theerror analysis 74.

It should be noted that conventional voltage-controlled oscillators arenot suitable for controlling the audio decoder 80 because they generatea clock signal which has too much phase jitter.

We claim:
 1. An apparatus for recovering audio information carried bymotion picture film in an analog soundtrack and in a digital soundtrack,said apparatus comprisingsample means for generating samples bysampling, output of an optical sensor sensing across widths of saidanalog soundtrack and said digital soundtrack, image means for forming atwo-dimensional image representation in response to said samplesgenerated along lengths or said soundtracks, and recovery means forrecovering said audio information in response to said two-dimensionalimage representation.
 2. An apparatus according to claim 1 furthercomprising a means for delaying audio information recovered by saidrecovery means.
 3. An apparatus according to claim 1 further comprisinganalog processing circuitry coupled to said optical sensor so as toreceive signals corresponding to said analog soundtrack, digitalprocessing circuitry coupled to said optical sensor so as to receivesignals corresponding to said digital soundtrack, and a switchswitchably coupled to outputs of said analog processing circuity andsaid digital processing circuitry, whereby audio information may beselectively recovered from either said analog soundtrack or said digitalsoundtrack.
 4. An apparatus according to claim 1 wherein said opticalsensor has a field of view extending beyond opposite outside edges ofsaid analog and digital soundtracks.
 5. An apparatus for transportingmotion picture film, said film carrying audio information in an analogsoundtrack and a digital soundtrack, said apparatus comprising anoptical sensor mounted in proximity to said film so as to have a fieldof view spanning widths of both soundtracks.
 6. An apparatus accordingto claim 5 further comprising a delay circuit coupled to an output ofsaid optical sensor.
 7. An apparatus according to claim 5 furthercomprising analog processing circuitry coupled to said optical sensor soas to receive signals corresponding to said analog soundtrack, digitalprocessing circuitry coupled to said optical sensor so as to receivesignals corresponding to said digital soundtrack, and a switchswitchably coupled to outputs of said analog processing circuity andsaid digital processing circuitry, whereby audio information may beselectively reproduced from either said analog soundtrack or saiddigital soundtrack.
 8. An apparatus according to claim 5 wherein saidfield of view extends beyond opposite outside edges of said analog anddigital soundtracks.
 9. An apparatus for transporting picture film, saidfilm carrying audio information in an analog soundtrack and a digitalsoundtrack, said apparatus comprisingsensor means for optically sensingsaid film across widths of both soundtracks, means for directing outputof said sensor means such that information sensed in said analogsoundtrack is directed to analog processing circuitry and informationsensed in said digital soundtrack is directed to digital processingcircuitry.
 10. An apparatus according to claim 9 further comprisingmeans for delaying said information sensed in said digital soundtrack.11. An apparatus according to claim 9 further comprising a switchswitchably coupled to outputs of said analog processing circuitry andsaid digital processing circuitry, whereby audio information may beselectively reproduced from either said analog soundtrack or saiddigital soundtrack.
 12. An apparatus according to claim 9 wherein saidsensor means has a field of view extending beyond opposite outside edgesof said analog and digital soundtracks.
 13. An apparatus according toclaim 5 further comprising analog processing circuitry coupled to saidoptical sensor so as to receive signals corresponding to said analogsoundtrack, and digital processing circuitry coupled to said opticalsensor so as to receive signals corresponding to said digitalsoundtrack.